130 research outputs found
Stochastic forward-backward and primal-dual approximation algorithms with application to online image restoration
Stochastic approximation techniques have been used in various contexts in
data science. We propose a stochastic version of the forward-backward algorithm
for minimizing the sum of two convex functions, one of which is not necessarily
smooth. Our framework can handle stochastic approximations of the gradient of
the smooth function and allows for stochastic errors in the evaluation of the
proximity operator of the nonsmooth function. The almost sure convergence of
the iterates generated by the algorithm to a minimizer is established under
relatively mild assumptions. We also propose a stochastic version of a popular
primal-dual proximal splitting algorithm, establish its convergence, and apply
it to an online image restoration problem.Comment: 5 Figure
Quasinonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods
Fixed point iterations play a central role in the design and the analysis of
a large number of optimization algorithms. We study a new iterative scheme in
which the update is obtained by applying a composition of quasinonexpansive
operators to a point in the affine hull of the orbit generated up to the
current iterate. This investigation unifies several algorithmic constructs,
including Mann's mean value method, inertial methods, and multi-layer
memoryless methods. It also provides a framework for the development of new
algorithms, such as those we propose for solving monotone inclusion and
minimization problems
Variable Metric Forward-Backward Splitting with Applications to Monotone Inclusions in Duality
We propose a variable metric forward-backward splitting algorithm and prove
its convergence in real Hilbert spaces. We then use this framework to derive
primal-dual splitting algorithms for solving various classes of monotone
inclusions in duality. Some of these algorithms are new even when specialized
to the fixed metric case. Various applications are discussed
Stochastic Quasi-Fej\'er Block-Coordinate Fixed Point Iterations with Random Sweeping
This work proposes block-coordinate fixed point algorithms with applications
to nonlinear analysis and optimization in Hilbert spaces. The asymptotic
analysis relies on a notion of stochastic quasi-Fej\'er monotonicity, which is
thoroughly investigated. The iterative methods under consideration feature
random sweeping rules to select arbitrarily the blocks of variables that are
activated over the course of the iterations and they allow for stochastic
errors in the evaluation of the operators. Algorithms using quasinonexpansive
operators or compositions of averaged nonexpansive operators are constructed,
and weak and strong convergence results are established for the sequences they
generate. As a by-product, novel block-coordinate operator splitting methods
are obtained for solving structured monotone inclusion and convex minimization
problems. In particular, the proposed framework leads to random
block-coordinate versions of the Douglas-Rachford and forward-backward
algorithms and of some of their variants. In the standard case of block,
our results remain new as they incorporate stochastic perturbations
- …